CHAPTER EIGHT

Case #15 FORECAST COMBINATION WITH LINEAR AND NONLINEAR TREND

 

Goal:  This case examines the practice of forecast combination stressing the role of diverse information as applied to forecasting the S&P 500 composite stock index.  Specifically, it examines:

  • Forecasting Trend using Linear Regression
  • Forecasting Nonlinear Trend with Multiple Regression
  • Testing conditions for Effective Forecast Combination
  • Estimating Optimal Weights in the Forecast Combination Process.

 

Problem Spreadsheet

 

For this case we selected a time series characterized by no seasonality and a non-linear trend.  Since stock prices are non-seasonal and follow exponential growth, we selected data on the S&P 500 Composite Stock Index (FSPCOM).  Our purpose is to stress test the forecast combination process by examining a case where forecast combination should fail to produce superior results!

 

The spreadsheet for this problem is C8_Case1.xls.  It contains the following data:

 

Variable

Data Range

FSPCOM

1947Q1-1994Q4

TIME

1947Q1-1994Q4

TIME_SQUARED

1947Q1-1994Q4

 

The series FSPCOM is quarterly data on the S&P 500 Composite Index.  The time indices will be used to estimate a linear and non-linear trend model. 

 

Examining Data for Stationarity

 

To examine the behavior of FSPCOM over the historical period (1947Q1-1994Q4), we generated a time-series plot of the data using Excel. 

 

 

Question #1: Based upon a time-series plot, do the quarterly data exhibit a non-linear trend?

 

ANSWER: Based upon the historical plot of quarterly data shown above, the trend is clearly non-linear as expected, since stock prices reflect compound rates of return.

 

Linear Trend Regression 

 

Using FORECASTXTM, we generated forecasts of FSPCOM for 1995 along with a holdout period for 1994 using a linear trend regression model.  Summary results are reported below.

 

 

Multiple Regression -- Result Formula

 

 

 

 

 

 

 

 

 

FSPCOM = -46.69  + ((TIME) * 1.82)

 

 

 

 

 

 

 

 

 

Forecast -- Multiple Regression Selected

 

 

 

 

 

Forecast

 

95% - 5%

95% - 5%

Date

Quarterly

Annual

 

Upper

Lower

Jan-1995

304.56

 

 

406.76

202.36

Apr-1995

306.38

 

 

408.59

204.17

Jul-1995

308.20

 

 

410.43

205.97

Oct-1995

310.02

1,229.16

 

412.26

207.77

Avg

307.29

1,229.16

 

409.51

205.07

Max

310.02

1,229.16

 

412.26

207.77

Min

304.56

1,229.16

 

406.76

202.36

 

Audit Trail -- Coefficient Table (Multiple Regression Selected)

 

 

 

Series

Included

 

Standard

 

 

Overall

Description

in Model

Coefficient

Error

T-test

Elasticity

F-test

FSPCOM

Dependent

-46.69

8.84

-5.28

 

524.56

TIME

Yes

1.82

0.08

22.90

1.36

 

 

Accuracy Measures

 

Value

AIC

 

 

2,123.60

BIC

 

 

2,126.86

Mean Absolute Percentage Error (MAPE)

63.83%

Sum Squared Error (SSE)

 

707,606.37

R-Square

 

 

73.41%

Adjusted R-Square

 

73.27%

Root Mean Square Error

 

60.71

 

Question #2: Evaluate the quality of the linear trend regression model.

 

ANSWER: The linear model appears to fit the data well as shown by the R-squared of .7341 and the large value of the F-statistic.  In addition, the coefficient on TIME is positive as expected, and significantly different from zero at the 99% level of confidence.  However, as shown by the plot of forecasts and actual data, the model completely misses the non-linear trend apparent in the data!  Accordingly, next we re-estimated the data using a non-linear trend regression model.

 

Non-Linear Trend Model

 

Using FORECASTXTM, we generated forecasts of FSPCOM for 1995 along with a holdout period for 1994 using a non-linear trend regression model.  Summary results are reported below.

 

 

Multiple Regression -- Result Formula

 

 

 

 

 

 

 

 

 

FSPCOM = 61.93  + ( (TIME) * -1.54 )  + ( (TIME_SQUARED) * 0.017406 )

 

 

 

 

 

 

 

Forecast -- Multiple Regression Selected

 

 

 

 

 

Forecast

 

95% - 5%

95% - 5%

Date

Quarterly

Annual

 

Upper

Lower

Jan-1995

413.15

 

 

477.09

349.20

Apr-1995

418.27

 

 

482.28

354.27

Jul-1995

423.40

 

 

487.47

359.33

Oct-1995

428.53

1,683.35

 

492.67

364.40

Avg

420.84

1,683.35

 

484.88

356.80

Max

428.53

1,683.35

 

492.67

364.40

Min

413.15

1,683.35

 

477.09

349.20

 

Audit Trail -- Coefficient Table (Multiple Regression Selected)

 

 

 

Series

Included

 

Standard

 

 

Overall

Description

in Model

Coefficient

Error

T-test

Elasticity

F-test

FSPCOM

Dependent

61.93

8.25

7.51

 

842.06

TIME

Yes

-1.54

0.20

-7.80

-1.15

 

TIME_SQUARED

Yes

0.02

0.00

17.58

1.67

 

 

Accuracy Measures

 

Value

AIC

 

 

1,937.56

BIC

 

 

1,940.82

Mean Absolute Percentage Error (MAPE)

48.23%

Sum Squared Error (SSE)

 

268,516.47

R-Square

 

 

89.91%

Adjusted R-Square

 

89.80%

Chi-Square

 

 

1.00

Cochrane-Orcutt

 

 

0.97

Mean Absolute Error

 

32.90

Mean Error

 

 

0.00

Mean Square Error

 

1,398.52

Normality Error

 

 

61.12

Root Mean Square Error

 

37.40

Standard Deviation of Error

 

37.49

Theil

 

 

11.95

 

Question #3: Evaluate the quality of the estimated non-linear trend model.

 

ANSWER:  The non-linear model appears to fit the data quite well as shown by the R-squared of .8991 and the large value of the F-statistic.  The coefficient on TIME is negative and significantly different from zero at the 99% level of confidence.  The coefficient on TIME_SQUARED is positive and significantly different from zero at the 99% level of confidence, consistent with the trend being non-linear.  The in-sample RMSE is 37.40, about a half of the linear trend model!

 

Can These Models be Combined?

 

To examine whether these two models can be combined, we estimate the following regression:

 

Audit Trail -- Coefficient Table (Multiple Regression Selected)

 

 

Series

Included

 

Standard

 

Overall

Description

in Model

Coefficient

Error

T-test

F-test

FSPCOM

Dependent

0.00

4.41

0.00

842.06

Linear_Forecast

Yes

0.00

0.06

0.00

 

Nonlinear_Forecast

Yes

1.00

0.06

17.58

 

 

Question #4: Based upon the estimated regression above, can we successfully combine these two forecasting methods without generating biased forecasts?

 

ANSWER:  As shown by the calculated t-statistic on the intercept term, we cannot reject the null of a zero intercept.  Accordingly, we can combine the two trend models without generating any forecast bias.

 

Optimal Forecast Combination

 

Next, we use multiple regression to calculate optimal forecast combination weights and forecasts for 1995.

 

Multiple Regression -- Result Formula

 

 

 

 

 

 

 

FSPCOM = 0.  + ( (Linear_Forecast) * 0. )  + ( (Nonlinear_Forecast) * 1.00 )

 

Audit Trail -- Coefficient Table (Multiple Regression Selected)

 

 

Series

Included

 

Standard

 

Overall

Description

in Model

Coefficient

Error

T-test

F-test

FSPCOM

Dependent

0.00

0.00

0.00

846.52

Linear_Forecast

Yes

0.00

0.06

0.00

 

Nonlinear_Forecast

Yes

1.00

0.06

17.63

 

 

Here, the optimal weight to the linear forecast is zero!  This should not be a surprise because the information about the linear trend is already contained in the non-linear model.  In fact, any other result would question the validity of this method of optimal combined forecasting weighting.

 

In-sample accuracy results for the combined model are reported below.

 

Accuracy Measures

 

Value

AIC

 

 

1,937.56

BIC

 

 

1,940.82

Mean Absolute Percentage Error (MAPE)

48.23%

Sum Squared Error (SSE)

 

268,516.47

R-Square

 

 

89.91%

Adjusted R-Square

 

89.80%

Chi-Square

 

 

1.00

Cochrane-Orcutt

 

 

0.97

Mean Absolute Error

 

32.90

Mean Error

 

 

0.00

Mean Square Error

 

1,398.52

Normality Error

 

 

61.12

Root Mean Square Error

 

37.40

Standard Deviation of Error

 

37.49

Theil

 

 

11.95

 

 

 

 

Method Statistics

 

Value

Method Selected

 

 

Multiple Regression

 

Forecasts for 1995 are reported below.

 

Forecast -- Multiple Regression Selected

 

 

Forecast

Date

Quarterly

Annual

Jan-1995

413.12

 

Apr-1995

418.23

 

Jul-1995

423.34

 

Oct-1995

428.45

1,683.14

Avg

420.78

1,683.14

Max

428.45

1,683.14

Min

413.12

1,683.14

 

A plot of FSPCOM and combined forecasts is shown below.

 

 

Question #5: Based upon the results above, do the combined forecasts outperform the other models? Explain.

 

ANSWER: As shown above, the weight given to the linear model is essentially zero.  Accordingly, there is no gain to combining the linear and non-linear trend models.

 

Indeed, the combined forecasts and statistics are exactly the same as those of the non-linear trend model.  The point:  For forecast combination to work you must employ differing information in the methods you combine!

 

Student Practice Question

 

Question #1: What improvement do you expect in forecast accuracy if we include the time index raised to the third power, i.e., allow a S-shaped trend?  Using FORECASTXTM, test your theory on the data of this case study.  Explain!